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Abstract 

This  paper  deseribes  a  research  study  whose  goal  is  to  understand  the  impaet  on  student  learning 
of  and  eareer  interest  in  Systems  Engineering  (SE)  through  a  set  of  diverse  pilot  SE  capstone 
experiences  being  implemented  in  eight  civilian  universities  and  six  military  academies.  The 
strategic  goal  addressed  by  this  researeh  is  to  explore  how  differing  course  designs,  structures 
and  materials  impaet  student  learning  and  eareer  interest  in  SE  in  order  to  augment  the  SE 
workforce  for  future  Department  of  Defense  (DoD)  and  related  industry  workforee  needs.  A 
work-in-progress,  this  research  outlines  the  rationale,  researeh  methods,  approaches  to  course 
organization,  structure,  and  delivery  used  across  14  diverse  institutions  with  SE  departments  at 
approximately  the  mid-point  of  eourse  delivery. 

Program  Rationale 

A  45%  growth  is  expected  in  SE  jobs  in  the  next  decade'  and  there  have  been  numerous  studies 
and  workshops  that  have  highlighted  the  shortfalls  in  both  the  number  and  eapability  of  the  SE 
workforce.  The  July  2006  National  Defense  Industrial  Assoeiation  (NDIA)  Task  Eorce  noted 
among  the  top  five  SE  issues  the  lack  of  adequate,  qualified  SE  human  capital  resources  within 
government  and  industry  for  alloeation  on  major  programs^.  In  the  July  2010  NDIA  white  paper 
on  eritical  SE  ehallenges.  Issue  2  was  identified  as;  The  quantity  and  quality  of  SE  expertise  is 
insufficient  to  meet  the  demands  of  the  government  and  defense  industry,  and  further  outlined 
certain  recommendations  to  build  SE  expertise  and  capacity.  In  particular,  it  recommended 
developing  SE  expertise  through  “role  definition,  seleetion,  training,  career  incentives,  and 
broadening  ‘systems  thinking’  into  other  diseiplines,”  and  made  a  number  of  specifie 
recommendations,  including  adding  an  introductory  course  in  SE  in  all  undergraduate 
engineering  and  technical  management  degree  programs;  and  working  with  major  universities  to 
reeommend  SE  curricula  to  improve  eonsisteney  aeross  programs  in  order  to  achieve 

-3 

standardization  of  skill  sets  for  graduates  . 

Research  Objectives  and  Program  Goals 

Research  on  Building  Education  &  Workforce  Capacity  in  Systems  Engineering,  (referred  to  as 
the  SE  Capstone  Project),  aims  to  understand  the  methods  through  which  SE  learning  and  career 
interest  may  be  increased  among  undergraduate  and  graduate  engineering  students.  The  key 
research  question  this  program  is  designed  to  address  is: 

What  organization  of  course  work  (course  sequenee,  course  materials,  faculty  characteristics, 
student  characteristics)  leads  to  the  largest  student  gains  in  (1)  SE  learning;  (2)  interest  in  SE 
eareers;  and  (3)  interest  in  DoD  problems  and  careers? 

This  researeh  is  being  condueted  in  the  context  of  14  “capstone”  courses,  in  most  cases  as  an 
integrative  eulminating,  project-based  course  involving  teams  of  students  working  together  on 
the  development  of  a  product  or  prototype  that  addresses  a  real  Department  of  Defense  (DoD) 
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need.  Implemented  as  pilot  eourses  in  eight  civilian  and  six  military  universities,  most  of  which 
are  members  of  a  SE-focused  University  Advanced  Research  Center,  or  UARC  based  at  Stevens 
Institute  of  Technology),  these  14  institutions  are  piloting  methods,  materials,  and  approaches  to 
create  new  courses  or  enhance  existing  courses  to  embed,  infuse,  and  augment  SE  knowledge,  as 
defined  by  the  Systems  Planning,  Research  Development,  and  Engineering  (SPRDE)-SE  and 
Program  Systems  Engineer  (PSE)  competency  model,  known  as  the  SPRDE-SE/PSE 
Competency  Model,  among  undergraduate  and  graduate  students.  Participating  university 
faculty  developed  new  course  materials  and  other  methods  and  strategies  to  recruit  and  provide 
substantive  SE  learning  experiences;  increase  exposure  to  authentic  DoD  problems,  such  as  low- 
cost,  low-power  computing  devices,  expeditionary  assistance  kits,  expeditionary  housing 
systems,  and  immersive  training  technologies. 

These  14  piloting  universities  are  implementing  methods  and  approaches  hypothesized  to  lead  to 
increased  student  interest  in  SE  education  and  careers,  particularly  in  DoD  and  related  industry 
contexts.  This  pilot  program  is  being  conducted  in  order  to  inform  the  development  of  a  national 
scale-up  effort  that  would  substantially  expand  the  number  and  capabilities  of  universities  that 
could  produce  SE  graduates  needed  for  the  DoD  and  related  defense  industry  workforce.  It  is 
anticipated  that  the  implementation  of  the  pilot  courses  will  lead  to  the  discovery  of  exemplary 
course  materials,  assessment  instruments,  and  other  lessons  that  will  be  deployed  to  accelerate 
the  adoption  of  effective  practices  and  materials  in  a  national  scale  up.  An  analysis  of  effective 
practices  based  on  the  experience  of  the  14  universities,  along  with  recommendations  for 
national  scale-up,  will  result  from  this  research. 

Literature  Review 

With  the  increased  focus  on  science  and  engineering  education  from  elementary  through  higher 
education  since  the  launching  of  the  SputniU ,  the  number  of  undergraduate  and  graduate  SE 
degree  programs  offered  in  the  United  States  over  the  past  50  years  has  reached  165  programs 
across  80  institutions^.  However,  the  best  methods  and  approaches  for  developing  SE  curriculum 
continue  to  evolve  as  the  discipline  evolves^'^.  In  the  past  decade,  in  an  effort  to  address 
workforce  development  needs  in  SE,  separate  competency  models  for  SE  have  been  developed 
by  individual  companies,  consortia,  and  professional  societies.  Descriptions  and  comparisons  of 
these  models  are  available  from  a  variety  of  sources'^'^^.  Over  the  last  few  years,  one  such 
model  development  took  place  in  the  defense  acquisition  community.  Subject  matter  experts 
developed  and  validated  the  SPRDE-SE/PSE.  The  SPRDE-SE/PSE  competency  model  is 
comprised  of  twenty-nine  areas  of  competencies  grouped  according  to  three  primary  “units  of 
competences”  -  analytical,  technical  management,  and  professional  -  as  shown  in  Table  1  . 

The  analytical  unit  covers  thirteen  competencies  related  to  the  technical  base  for  cost  and  aspects 
of  the  system  life  cycle.  The  technical  management  unit  addresses  twelve  competencies  on  the 
technical  side  of  project  management,  and  the  professional  unit  covers  the  broader  competencies 
of  communication,  problem  solving,  systems  thinking  and  ethics. 


Table  1:  SPRDE-SE/PSE  Competency  Model 


Analytical 

(13) 

1.  Technical  Basis  for  Cost 

2.  Modeling  and  Simulation 

3.  Safety  Assurance 
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4.  Stakeholder  Requirements  Definition 
(Requirements  Development) 

5.  Requirements  Analysis  (Eogieal  Analysis) 

6.  Arehiteetural  Design  (Design  Solution) 

7.  Implementation 

8.  Integration 

9.  Verifieation 

10.  Validation 

11.  Transition 

12  System  Assuranee 

13.  Reliability,  Availability,  and  Maintainability 

Teehnieal 

Management 

(12) 

14.  Deeision  Analysis 

15.  Teehnieal  Planning 

16.  Teehnieal  Assessment 

17.  Configuration  Management 

18.  Requirements  Management 

19.  Risk  Management 

20.  Teehnieal  Data  Management 

21.  Interfaee  Management 

22.  Software  Engineering 

23.  Aequisition 

24.  SE  Eeadership 

25.  System  of  Systems 

Professional 

(4) 

26.  Communieations 

27.  Problem  Solving 

28.  Strategie  Thinking 

29.  Professional  Ethies 

In  general,  eompeteney  models  are  used  not  only  for  workforee  seleetion  and  development,  but 
also  for  edueational  purposes.  In  partieular,  SE  eompeteney  models  are  beeoming  more  widely 
used  in  support  of  SE  workforee  development,  edueation,  and  training This  researeh  task 
draws  upon  the  SPRDE-SE/PSE  eompeteney  model  to  define  learning  objeetives  for  a  set  of 
eourses  being  implemented  aeross  the  14  piloting  universities. 

Program  Structure  and  Research  Design 

This  pilot  study  is  being  implemented  using  a  mixed  methods  approaeh  in  three  sequential 
phases  implemented  over  a  19-month  period.  Phase  1  (Mareh  1,  2010-May  15,  2010)  launehed 
the  Planning  and  Startup  effort  to  identify  the  seleetion  eriteria  and  proeess  for  ehoosing  the 
partieipating  institutions  and  to  identify  the  requirements  eaeh  would  need  to  implement  as  part 
of  the  researeh.  These  ineluded  a  set  of  required,  “eommon  assessments”  that  would  allow 
analysis  of  student  outeomes  aeross  all  projeets,  as  well  as  projeet  teams’  requirement  to  develop 
an  aetual  produet,  prototype  or  other  artifaet  based  on  a  real  DoD  problem.  A  set  of  four 
problem  areas  was  provided  by  the  sponsor  (Table  2). 


SERC  RT19_McGRATH_STEVENS_201 1ASEE_VANCOUVER,CANADA_201 1  JUNE 


Page  3 


Table  2 

1 .  Low-cost,  low-power  computers  leveraging  open-source  technologies  and  advanced  security  to 
support  sustainable,  secure  collaboration; 

Portable,  renewable  power  generation,  storage,  and  distribution  to  support  sustained  operations  in 
austere  environments  and  reduce  dependency  on  carbon-based  energy  sources; 

Portable,  low-power  water  purification; 

2.  An  expeditionary  assistance  kit  around  low-cost,  efficient,  and  sustainable  prototypes  such  as  solar 
cookers,  small  and  transportable  shelters,  deployable  information  and  communication  technologies, 
water  purifiers,  and  renewable  energies.  These  materials  would  be  packaged  in  mission-specific 
HA/DR  kits  for  partner  nation  use; 

3.  Develop  modular,  scalable,  expeditionary  housing  systems  that  possess  "green"  electric  power  and 
water  generation,  waste  and  wastewater  disposal,  hygiene,  and  food  service  capabilities.  Systems 
should  be  designed  to  blend  in  to  natural/native  surroundings  and  with  minimal  footprint; 

4.  Continued  investigation  and  exploration  into  the  realm  of  the  possible  with  respect  to  “Immersive” 
training  technologies.  Objective  is  to  flood  the  training  audience  environment  with  the  same 
STIMULI  that  one  would  experience  during  actual  mission  execution.  Where  possible  lull  sensory 
overload  is  desired  much  the  same  as  experienced  in  combat.  Specific  S&T  areas  for  development 

■  Virtual  Human.  Successful  modeling  of  emotions,  speech  patterns,  cultural  behaviors, 
dialogue  and  gestures. 

■  Universal  Language  Model.  The  ability  for  trainees  to  seamlessly  converse  with  the  Virtual 
Human. 

■  Virtual  Character  Grab  Controls.  The  ability  for  exercise  controllers  to  assume  control  of 
virtual  characters. 

■  Automated  Programming.  Cognitive  learning  models  and  the  ability  for  exercise  controllers 
to  adjust  virtual/live  simulations. 

■  Low  cost  wireless  personnel  sensors. 

■  Sensors  (i.e.,  lightweight  vests)  that  facilitate  physical  stimuli  (i.e.,  wounds,  shots)  to 
trainees. 


Phase  2/Pilot  Implementation  (May  15,  2010-June  30,  2011)  has  focused  on  the  development  of 
course  materials  and  customized  assessments,  implementation  of  the  courses,  and  evaluation  of 
the  courses  at  the  14  piloting  institutions.  Implementation  ranges  from  a  one-semester  “broad 
spectrum”  introduction  to  SE  course  to  a  two-semester  senior  design  course,  to  a  supplemental 
independent-study  approach  that  augments  a  traditional  senior  design  course  with  SE  modules. 

During  Phase  2,  each  of  the  institutions  recruited  student  teams,  organized  the  course  structure 
(materials  and  faculty  participation),  identified  and  initiated  collaborations  with  DoD  and 
industry  mentors  who  would  serve  in  a  variety  of  roles  with  student  teams,  developed  and 
administered  their  own  course  assessments  based  on  the  learning  objectives  of  their  particular 
courses,  and  are  delivering  the  courses. 

Near  the  beginning  of  Phase  2,  the  DoD  arranged  an  orientation  session  for  SE  Capstone  faculty 
to  meet  DoD  representatives  to  learn  more  about  the  authentic  DoD  problem  areas  being  pursued 
in  student  projects.  Problem  sets  were  discussed  and  arrangements  were  made  to  structure  the 
process  for  ongoing  collaborations. 
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Key  SE  concepts  to  be  assessed  across  all  pilots  are  drawn  from  the  SPRDE-SE/PSE 
Competency  Model  and  include: 

•  Understanding  and  definition  of  the  problem/need  to  be  addressed  by  a  systems  solution 
including  its  context  and  related  life-cycle  aspects 

•  System  definition  (ConOps/System  requirements),  including  the  necessary  trade-offs 
between  conflicting  requirements 

•  System  architecture  and  design  and  the  corresponding  structural/technical  trade-offs 

•  System  integration,  verification  and  validation  including  the  trade-offs  between  coverage  and 
available  time  and  resources. 

Phase  3/Analysis,  Recommendations  &  Dissemination  (July  1,  2011  -  September  30,  2011)  will 
analyze  the  results  from  all  14  participating  universities  and  integrate  them  into  a  single  set  of 
observations  to  the  sponsor  about  the  effectiveness  of  the  pilot  programs,  analysis  of  pre-/post 
learning  of  SE  content,  skills,  and  career  interest,  and  the  degree  to  which  learning  outcomes 
were  achieved.  It  will  also  create  a  set  of  recommendations  on  how  to  scale-up  the  pilot  program 
to  be  conducted  across  the  U.S. 

Currently  in  Phase  2,  approximately  300  undergraduate  and  graduate  students  are  or  have  been 
involved  in  SE  Capstone  pilot  courses.  To  date,  one  university  has  completed  the  SE  Capstone 
course  and  the  remaining  universities  will  conclude  the  pilot  course  at  the  end  of  the  spring  2011 
semester. 

Research  Design  and  Assessment  Instruments 

As  part  of  the  pilot,  each  university  team  was  required  to  develop  its  assessments  tailored  to  the 
specific  learning  objectives  of  its  course.  In  addition,  each  university  was  required  to  administer 
three  common  assessments  to  all  participating  students. These  would  be  used  to  gauge  student 
learning  of  specific  SE  concepts  and  skills  and  would  allow  not  only  comparisons  across 
universities  but  would  provide  data  for  correlating  different  outcomes  with  differences  in 
implementation. 

These  assessments  aimed  to  measure  the  following  student  outcomes: 

•  Understanding  of  what  the  discipline  of  SE  is 

•  Understanding  what  systems  engineers  do 

•  Understanding  the  qualities  and  skills  that  systems  engineers  bring  to  projects 

•  Consideration  of  a  career  in  SE 

•  Development  and  practice  of  the  skills  of  systems  engineering 

•  Understanding  how  systems  engineers  think  (analytic  skills) 

As  a  condition  of  participation,  all  universities  are  required  to  administer  three  required, 
“common”  assessments  designed  to  measure  SE  learning,  career  interest,  and  awareness  and 
interest  in  DoD  problems  and  careers.  In  addition  to  these  assessments,  the  research  team  is  also 
collecting  data  from  Pis  and  from  DoD  and  industry  mentors  working  with  student  teams. 
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The  common  assessments  to  measure  student  progress  toward  stated  learning  objectives  were 
developed  and  are  being  administered  to  more  than  300  undergraduate  and  graduate  students 
during  the  2010-11  academic  year.  These  student  assessments  include: 

(1)  pre-  and  post-surveys  to  gauge  knowledge  of  SE,  interest  in  SE  careers,  and  awareness  of 
a  spectrum  of  DoD  SE  problems; 

(2)  a  pre-  and  post-  case  study  analysis  of  an  SE  problem;  and 

(3)  required  weekly  student  blog  posts  to  measure  student  progress  toward  more 
sophisticated  SE  analysis  in  the  context  of  their  own  Capstone  projects. 

Data  on  course  materials,  course  organization,  customized  assessments  from  participating 
universities,  student  demographics,  and  type  of  institution  are  also  being  collected  and  analyzed, 
as  well  as  surveys  from  faculty,  DoD  mentors,  and  industry  representatives. 

Common  assessment  #1,  the  student  survey,  gathers  information  about  students’  backgrounds, 
interests,  and  prior  experience  with  SE,  asks  them  about  their  career  aspirations,  and  then  asks 
them  five  open-ended  questions  designed  to  assess  their  understanding  of  what  SE  is  and  of  SE 
careers.  The  post-implementation  version  of  this  survey  contains  these  five  open-ended  questions 
and  also  includes  several  satisfaction  questions  about  the  course.  The  five  open-ended  questions 
were: 

•  What  is  SE?  Define  it  as  best  you  can. 

•  In  the  context  of  SE,  give  an  example  of  a  system  that  would  be  addressed  by  a  systems 
engineer. 

•  How  might  systems  engineers  differ  from  disciplinary  engineers  (mechanical,  structural, 
etc.)  working  on  a  multidisciplinary  team?  Give  an  example. 

•  Eist  five  words  that  describe  the  skills  and  qualities  that  might  be  needed  by  a  systems 
engineer. 

•  Name  three  engineering  problem  areas  that  you  think  are  currently  being  addressed  by 
the  DoD. 

The  second  assessment  was  designed  to  see  the  extent  to  which  students  had  integrated  SE 
concepts  into  their  thinking  by  asking  them  to  transfer  that  thinking  to  another  situation.  The 
students  were  presented  with  the  story  of  the  Bradley  Eighting  Vehicle,  summarized  and 
including  a  few  short  clips  from  The  Pentagon  Wars — an  amusing  fictionalized  film  version  of 
the  book  by  the  same  name.  Students  were  asked  to  read  and  discuss  the  scenario  and  then 
respond  individually  to  a  single  prompt:  “Could  the  problems  encountered  in  developing  the 
Bradley  Eighting  Vehicle  have  been  avoided?  Explain  your  answer.”  We  provided  the  faculty 
with  a  link  to  the  results  and  encouraged  them  to  use  these  to  begin  a  discussion  of  the  role  of 
systems  engineers  in  large-scale  complex  projects  such  as  this.  The  same  material  was  presented 
as  a  post-test,  with  the  expectation  that  the  responses  would  be  more  detailed  and  would  show 
greater  evidence  of  SE  thinking  after  the  capstone  experience. 
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The  third  assessment  was  designed  to  provide  a  window  into  the  progress  of  student  learning 
during  the  eourse,  ineluding  the  SE  eompeteneies  addressed.  It  required  the  students  to  write 
weekly  blogs  that  summarized  their  experienee  for  the  week.  The  blog  prompts  were: 

•  What  did  you  and  your  group  aecomplish  this  week? 

•  Which  SE  competencies  best  align  with  what  you  did  this  week? 

•  What  specifically  did  you  do  in  terms  of  each  of  the  competencies  you  listed? 

The  blog  posts  were  thus  less  of  an  assessment  per  se  and  more  of  a  way  to  provide  both  the 
project  team  and  the  DoD  advisors  with  information  about  project  teams’  progress.  It  was  also 
hoped  that  the  DoD  advisors  would  use  the  blogs  as  one  means  of  communicating  with  the 
students.  The  prompts  for  the  final  blog  prompt,  at  the  end  of  the  course,  were  more  elaborate 
and  reflective.  They  were.' 

•  What  were  the  most  important  system-level  trade-offs  you  had  to  consider  during  this 
project? 

•  If  you  were  to  start  this  project  over  again,  what  would  you  do  differently? 

However,  the  only  school  to  complete  the  project  in  one  semester  and  therefore  the  first  to  finish 
wanted  the  final  blog  to  be  more  comprehensive  and  so  replaced  the  above  with  the  following  set 
of  questions,  which  we  may  ask  the  other  projects  to  use  as  well: 

•  Do  you  feel  that  what  you  learned  about  SE  benefited  your  capstone  design  project? 

•  Do  you  think  SE  provides  a  different  approach  to  engineering  design  than  what  you  were 
familiar  with?  Why  or  why  not? 

•  Describe  the  advantages  and  disadvantages  of  working  in  multidisciplinary  teams  on  a 
capstone  design  project.  What  tasks  and  processes  were  difficult  for  you  to  execute  and 
what  were  you  least  prepared  to  deal  with?  Please  explain. 

•  What  advice  would  you  give  a  peer  on  why  it  is  important  to  take  a  course  that 
incorporates  SE  principles  within  the  context  of  a  design  project? 

•  Now  that  you  have  been  introduced  to  SE  principles  and  how  they  apply  to  a  project,  do 
you  think  you  would  like  to  have  a  career  in  SE?  Why  or  why  not? 

In  addition  to  the  student  surveys  and  other  student  assessments,  several  collections  of  data  will 
be  made  throughout  the  courses  from  Pis  and  DoD  and  industry  mentors. 

Course  Structures  and  Foci 

A  majority  of  the  piloting  universities  relied  on  the  expertise  of  SE  faculty  to  lead  or  contribute 
to  the  conceptualization,  development,  and  implementation  of  the  program  and  course  materials. 
Beyond  this  similarity,  however,  were  many  differences  among  the  piloting  universities, 
providing  a  diverse  array  of  methods,  approaches  and  structures  for  the  implementation  of  SE 
Capstone  courses.  Table  3  summarizes  the  course  structures,  DoD  problem  area(s)  addressed, 
students  impacted,  and  other  characteristics  of  the  portfolio  of  14  pilot  projects. 
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Table  3 

Institution  C 

ourse/ 

Project 

Description 

DoD  Problem  Area  Focus 

Students 

Participating 

Auburn 

University 

SE 

in  a  Secure 

Computing 

Intensive 

Environment 

course  [Eall  2010]  is  a 
broad-spectrum  overview  to 
SE.  It  introduces  major 
concepts  using  a  case  study 
of  the  security  architecture 
of  two  open  systems  under 
consideration  by  DoD. 

course  [Spring  2011]  is 
an  actual  project  employing 
low-cast,  open-source, 
computing.  The  students 
will  demonstrate  secure 
collaboration  using  the 
Android  open  source 
software  stack. 

Eow-cost,  low-power  computers 
leveraging  open-source  technologies  and 
advanced  security  to  support  sustainable, 
secure  collaboration. 

Course  material  for  the  U*  course  will 
be  delivered  through  presentations 
by  speakers  from  industry  and 
government;  lectures,  and  interactive 
students  activities. 

The  2“‘*  course  is  a  hands  on  sequel 
in  which  students  will  complete  their 
Defense-focused  capstone  project 

33 

3  undergrads 

30  grads 

Mix  ofCS,  IE, 
and  EE 

On-campus 
and  distance 
education 

Missouri 

S&T 

University 

Agile  Systems 
Engineering- Active 
and  Experiential 
Eearning  Approach 

U‘ Course  [Eall2010]; 
Introduction  to  SE  provides 
the  student  with  basic 
understanding  of  main 
concepts,  tools,  and 
processes  of  SE. 

2“‘*  Course  [Spring2011]; 
Physical  Artifact  Creation 
and  Validation. 

Development  of  detailed 

Immersive  Training  Technologies. 

Subtle  simulation  of  real  battlefield 
scenarios.  Operational  scenarios  simulate 
getting  shot,  getting  hit,  and  minor 
restriction. 

10  undergrads 

27  grads 

Mix  ofECE, 

ME,  and  AE 

On-campus  and 

distance 

education 
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design  for  a  wireless  haptic 
vest  with  embedded 
sensors.  Students  will  focus 
on  the  wireless  tech  to 
activate  embedded  sensors 
and  mechanical  components 

Penn  State 
University 

Interdisciplinary 
Capstone  Design 
Project 

This  is  a  one-semester 
course/project  [Eall2010]. 
Modules  delivered  by  SE 
faculty.  Projects  are 
completed  using  the 

Bernard  M.  Gordon 

Beaming  Eactory,  a  lab 
providing  modern  design, 
prototyping,  and 
manufacturing  facilities. 

Expeditionary  Assistance  Kit. 

1 .  Water  purification  system 

2.  Power  generation  from  renewable 
energy  sources 

3.  Eocal  situational  awareness  system 

4.  Global  low-bandwidth 
communication  unit 

17  undergrads 

Mix  of  BE,  CE, 
EE,  ME,  IE 

Southern 

Methodist 

University 

Leveraging 

Interdisciplinary 

Teaching 

Environments  to 
Research  Immersive 
Training 

Environments 

U' Course  [Eall2010], 
students  work  in 
interdisciplinary  teams  to 
design  an  architecture 
solution  that  meets 
customer  specifications. 

2“‘*  Course  [Spring  2011], 
students  will  continue  to 
work  on  interdisciplinary 
teams  to  build  and  test  a 
prototype  of  their  design. 

Immersive  Training. 

The  objective  is  to  improve  existing 
capabilities  in  three  areas:  (1)  fidelity  of 
motion  capture  systems,  (2)  reduction  of 
infrastmcture  required  for  team-based 
motion  capture,  and  (3)  high  resolution 
facial  expression  capture  and  replication 

1 1  undergrads 

Mix  ofCS,  EE. 
and  ME 

3  PhD  students 
contribute  on 
course  design 
and  facilitation 

Stevens 
Institute  of 
Teehnology 

Building  Education 
and  Workforce 
Capacity  in  SE 

Implementation  of  SE  in 
capstone  senior  design 
course  [one  year  long] 

Green-Expeditionary  Housing. 

Eor  a  100  person  EOB  and  3-6  months 

24 

SERC  RT19_McGRATH_STEVENS_201 1ASEE_VANCOUVER,CANADA_201 1  JUNE 


Page  9 


through  Capstone 
Design 

A  series  of  6  SE  all-day 
workshops,  are  being 
delivered  to  introduce  SE 
principles  and  methods  to 
all  students. 

Workshops  take  place  every 
4-6  weeks. 

deployment.  Modular  housing  with 
micro-grid  support  for  alternate  energy 
sources,  including  low  impact  solutions 
for  waste  and  water 

EM,  ME,  EE, 

CE,  Civ  Eng, 

A&T 

University 
of  Maryland 

Special  Topics  in  SE 

This  is  a  one  semester 
course  that  is  offered  twice 
over  one  academic  year. 

The  goal  of  both  graduate 
and  undergraduate  pilots  is 
to  introduce  students  to  SE 
via  a  hands  on  project 
experience 

Eocuses  on  low-cost,  low-power 
computers  leveraging  open  source 
technologies.  Supports  integrated 
wireless  sensor  networks,  vehicle  bus,  and 
black  box 

15  undergrads 

Erom  all 

engineering 

departments 

University 
of  Virginia 

Extensible  SE 

Capstone  Experience 
for  Non-SE  Seniors 

It  exposes  students  to  the 
entire  SE  process.  This  will 
be  accomplished  via  two 
interdisciplinary  capstone 
projects  over  one  academic 
year. 

During  the  semester  the 

two  teams  will  test  and 
evaluate  each  other’s 
projects. 

(1)  This  project  involves  Immersive 
Training  Capability,  Combining  Sensors, 
Actuators  and  Virtual  Environ  to  Track 
Hand  and  Einger  Position 

(2)  This  project  will  develop  a  robust 
decision  support  system  for  rapid  of  water 
supply  and  sanitation  technologies  for 
HA/DR  operations. 

19  undergrads 
from  all 
engineering 
departments 

2  SE  grad 
students  serve  as 
TAs. 

Wayne  State 
University 

Integrated  Material 
Design  and 

Realization  for 

This  project  integrates  SE 
product  development 
concepts  across  4  courses  at 

Expeditionary  Operations. 

The  projects  will  be  focused  on 

30 

Mix  of  ME,  IE 
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HA/DR  Kits 

the  undergrad  and  graduate 
level. 

1  Pull  semester  eourse 
(Winter  2011)  plus  modular 
insertion  into  multiple  other 
courses  (start  process  -  Sept 
2010) 

development  of  elements  of  HA/DR  kits, 
sueh  as  solar  oven,  water  purifieation 
system,  alternative  energy 

and  SE 

Air  Force 
Institute  of 
Teehnology 

Introduction  to  SE 
Process  and  Design 

This  eourse  [one  academic 
year]  provides  a  broad 
introduction  to  a  systematic 
approach  necessary  for  the 
formulation,  analysis, 
design  and  evaluation  of 
complex  systems. 

Eow-power  computing  for  operations  in 
austere  environments. 

Development  of  a  novel  hybrid  electric 
UAV  for  near  silent,  long  loiter,  low 
energy  operations. 

5  grads 

Mix  of  AE,  SE 

Naval  Post- 

Graduate 

Sehool 

Transforming 

Graduate  Education 
in  SE 

A  series  of  8  core  SE 
courses  [one  academic  year] 
in  the  master’s  curriculum 
are  being  taught  in  a  faeulty 
team-based  pedagogy,  with 
the  eapstone  project 
integrated  into  the  entire 
currieulum  as  a  carry- 
through,  hands-on 
experience.  The  courses 
provide  a  holistie  span  of 
education  from  systems 
thinking,  quantitative 
analysis,  through  system 
design  and  production 

Expeditionary  Operations  and  HA/DR 
Assistance  Kits. 

Development  of  novel,  low  density  power 
supplies,  advanced  materials  with  low 
thermal  and  visibility  properties,  low 
signature  communication  devices. 

25  grads 

SE 

US  Air 

Force 

Academy 

Capstone  Design 
Projeet 

This  project  integrates 
sequentially  two  SE  courses 
over  one  academic  year. 

Eow  Power  Computing 

A  10  KVA  solar  energy  system  for 

7  undergrads 
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Students  will  learn  to 
successfully  work  in  a 
multidisciplinary  team,  to 
apply  SE  and  management 
tools,  communicate  project 
details,  and  evaluate 
contemporary  military 
issues 

deployed  operations.  The  end  system  will 
incorporate  smart  grid  technology  to 
facilitate  control  and  integration 

Mix  of  EE,  CE 

US  Naval 
Academy 

Principles  of 
Engineering  Systems 
Design 

The  senior  design  capstone 
course  [one  academic  year] 
is  enhanced  with  additional 
SE  sections  based  on 
experimental  coursework. 
This  is  an  independent 
study  course  based  on 
Defense  Acquisition 
University  courses 

Expeditionary  Ops. 

Portable,  low  power  water  purification. 
Portable,  renewable  power  generation, 
storage  and  distribution 

16  undergrads 

Mix  of  EE,  CE, 
NA,  OE 

US  Military 
Academy  at 
West  Point 

Systems  and 
Engineering 
Management  Design 

This  capstone  course  [one 
academic  year]  emphasizes 
SE  in  technology-based 
organizations.  Cadets 
examine  interconnections 
between  planning, 
organizing,  leadership, 
control,  and  the  human 
element  in  production, 
research  and  service 
organizations 

Immersive  Training 

Augmented  Reality:  synthetic  environ, 
decision  analysis  for  optical  &  video 
displays,  high  fidelity  tracking 

4  undergrads 

Mix  of  SE,  EM, 
and  OR 

US  Coast 

Guard 

Academy 

SE  Capstone 
Enhancement 

This  course  [one  academic 
year]  incorporates  critical 
elements  of  SE.  Cadets  will 
have  regular  contact  with 

Expeditionary  Ops.  Green  Power 
Generation  HA/DR 

Portable  hull  inspection  system. 

54+ 

undergrads 

Mix  of  Civ  Eng, 
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external  customers  [USCG 
Shore  Maintenance 

Command  and  USCG 
Aviation  Eogistics  Center] 
and  must  defend  their  work 
at  preliminary  and  final 
design  reviews 

Green  electric  power  in  remote  hot 
climates. 

In  water  remote  propeller  cleaner. 
Hybridization  system  for  fleet  vehicles. 

EE,  and  ME 

NOTES: 

1 .  Sources  include  proposal  documents  submitted  by  universities,  first  progress  reports,  and  a  summary  prepared  for  the  2010 
Annual  SERC  Research  Review  conference  Nov.  9-10,  2010. 

2.  The  number  of  students  shown  in  the  table  above  include  only  those  who  are  directly  involved  in  the  entire  SE  Capstone 
course  [coursework  -l-project]. 

3.  Abbreviations  : 

EM:  engineering  management,  CE:  computer  engineering.  Civ  Eng:  civil  engineering,  EE:  electrical  engineering,  NA:  naval 
architecture,  OE:  ocean  engineering,  AE:  aerospace  engineering,  A&T:  arts  and  technology,  OR:  operations  research,  IE: 
industrial  engineering,  ME:  mechanical  engineering,  CS:  computer  science,  SE:  systems  engineering,  BE:  biomedical 
engineering 
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Preliminary  Observations  and  Lessons  Learned 

An  interim  report  was  submitted  by  each  piloting  university  in  January  2011.  The  following 
section  summarizes  the  progress,  status,  and  preliminary  observations/findings  based  on,  in  all 
but  a  few  cases,  the  mid-point  of  course  implementation. 

The  key  features  that  differentiated  the  organizational  structure  of  the  programs  at  the  different 
Institutions  of  Higher  Education  (IHEs)  were  the  following; 

•  Eaculty:  The  collaboration  of  two  or  multiple  faculty  members  on  course  design  and 
implementation.  At  1 1  institutions,  faculty  came  from  at  least  three  separate  engineering 
disciplines,  literally  embodying  the  multi-disciplinarity  of  a  SE  team.  SE  faculty  were  the 
largest  percentage  of  participating  faculty. 

•  Courses:  The  integration  of  the  SE  component  into  existing  courses  or  the  creation  of  entirely 
new  courses. 

•  Course  sequencing:  The  implementation  of  a  course  sequence  that  included  an  introductory 
course  followed  by  a  capstone  experience  or  a  capstone  experience  only. 

•  Student  population:  The  involvement  of  either  undergraduates  or  graduate  students  as 
learners^  or  a  mixed  class  with  both  undergraduates  and  graduate  students. 

•  Mentors;  The  presence  and  level  of  active  and  meaningful  involvement  of  DoD  and  industry 
mentors  in  a  variety  of  student  learning  experiences. 

The  ultimate  measure  of  effectiveness  of  the  course  design  will  be  determined  by  the  student 
outcomes  identified  as  project  goals: 

•  Increase  student  learning  of  SE  competencies 

•  Increase  student  interest  in  DoD  problems/careers 

•  Increase  career  interest  in  SE  study  and  careers 

It  is  premature  at  this  stage  of  course  implementation  to  correlate  student  outcomes  with  the 
structure  or  content  of  the  courses  or  with  any  particular  strategies  or  course  materials.  However, 
the  Pis’  interim  reports  provide  some  insights  into  preliminary  lessons  learned  about  course 
objectives  and  implementation.  These  observations  fall  into  several  categories: 

1 .  Challenges  teaching  the  broad  topic  of  SE  to  non-SE  majors  under  time  constraints. 

2.  Challenges  with  equivalent  grading  policies  in  multi-disciplinary  teams,  particularly 
where  SE  was  an  overlay  to  an  existing  multi-disciplinary  team  structure. 


'  Some  institutions  engaged  graduate  students  as  teaching  or  technical  assistants,  but  here  we  are 
referring  to  graduate  students  who  are  participating  in  the  course  for  credit. 
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3.  Challenges  with  content-domain-specific  problem  areas  and  with  finding  meaningful 
ways  for  other  disciplinary  majors  to  contribute. 

4.  Motivating  external  mentors  to  bring  authentic  professional  experiences  to  the  learning 
experience  and  sustaining  that  involvement  over  time. 

5.  SE  content  modules  provide  opportunities  to  bring  non-SE  majors  to  a  common 
understanding.  These  have  been  implemented  with  varying  frequency,  durations,  and 
numbers  across  several  projects. 

6.  Where  possible,  the  integration  of  Defense  Acquisition  University  modules  provide 
additional  incentives  for  students  to  gain  desirable  certifications,  but  scheduling  common 
experiences  for  students  to  take  these  modules  impacted  completion. 

In  some  cases,  the  challenges  identified  were  common  to  the  formation  and  operation  of 
effective,  multi-disciplinary  teams. 

7.  Provide  subject  matter  expertise  (internal  and  external)  to  infuse  sufficient  disciplinary 
knowledge  such  that  students  may  focus  on  the  bigger  SE  competencies. 

DoD  Problem  Area  Addressed 


Figure  1  illustrates  each  university’s  choice  of  one  or  more  problem  areas  based  on  existing 
faculty  expertise  and  interest.  Two  civilian  universities  and  one  service  academy  chose  to  work 
in  more  than  one  of  the  DoD  problem  areas.  More  than  half  the  projects  (8)  addressed  DoD 
Problem  Area  1  (low-cost/low-power  computers);  Problem  Area  4  (immersive  training 
technologies)  was  the  next  most  popular  choice;  with  the  remaining  two  areas  divided  among  the 
other  partner  institutions: 


€0% 

50% 

40% 

30% 

20% 

10% 

0% 


DoD  Problem  Areas  Addressed 


h’roblem  Area  1:  Low- 
cost,  lew-power 
computers 


Hrob  em  Area  4: 
Immersive  training 
technologies 


Problem  Area 2: 
Expediticnary 
assistance  kit 


Problem  Area  i: 

Expeditionary 
housing  systems 


Ceriesl 


57.10% 


28.60% 


14.30% 


21.40% 


Figure  1 
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Problem  areas  were  chosen  because  faculty  members  with  expertise  in  a  particular  area  were 
available  and  interested;  because  the  problem  area  was  considered  to  have  human  interest  or 
related  to  a  social  concern  and  would  therefore  attract  students;  or  because  the  institution  had 
ongoing  projects  in  that  area.  Only  one  PI  reported  a  concern  about  working  on  a  DoD  problem, 
and  that  was  only  from  one  student. 

Faculty  Involvement 

A  majority  of  the  universities  relied  on  the  expertise  of  SE  faculty  to  lead  or  contribute  to  the 
conceptualization,  development,  and  implementation  of  the  program,  but  many  other  faculty 
were  involved  as  well,  particularly  from  mechanical  engineering  and  computer  science.  At  1 1 
institutions,  faculty  came  from  at  least  three  separate  engineering  disciplines,  literally  embodying 
the  multi-disciplinarity  of  an  SE  team.  Figure  2  represents  the  percentage  of  the  14  pilot 
universities  that  included  those  types  of  disciplinary  faculty  in  the  RT-19  project: 


Figure  2 


As  indicated  in  Figure  3,  nearly  two-thirds  of  the  fourteen  projects  were  planned  and 
implemented  by  teams  of  two  or  three  faculty  members,  but  four  projects  included  four  or  more 
faculty.  Only  one  institution  developed  a  capstone  course  that  was  planned  and  taught  by  a  single 
faculty  member: 
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Faculty  teams 


■  2  to  3  faculty  members 
per  institution 

■  4  or  more 

□  1  faculty  member 


Figure  3 


Faculty  took  on  different  roles,  ineluding  that  of  elassroom  instruetor,  eurrieulum  developer, 
projeet  advisor,  and  SE  subjeet  matter  expert,  with  some  being  several  of  these.  Figure  4 
represents  the  pereent  of  all  faeulty  in  the  projeet  where  faeulty  eould  play  more  than  one  role. 


Course  Sequenees,  Struetures,  and  Types  of  Student 


There  was  a  diverse  array  of  methods,  approaehes  and  struetures  for  the  implementation  of  the 
eourses.  The  table  on  the  following  page  summarizes  the  differenees  in  type  of  student 
(graduate/undergraduate/both),  eourse  integration,  and  type  of  eourse  sequenee. 


Students: 

Undergraduates/ 

Graduate 

Integrated  into 
Existing  SE 
Courses 

Intro  course  + 
Capstone/ 
Capstone 
only 

Auburn  University 

U 

N 

I/C 
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Missouri  S&T  University 

U/G 

Y 

C 

Penn  State  University 

U 

Y 

C 

Southern  Methodist  University 

U 

Y 

I/C 

Stevens  Institute  of  Technology 

U/G 

Y 

c 

University  of  Maryland 

U 

Y 

c 

University  of  Virginia 

U 

Y 

c 

Wayne  State  University 

U/G 

Y 
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All  but  one  institution  integrated  the  RT-19  effort  into  existing  SE  eourses,  with  only  Auburn 
developing  a  completely  new  course.  Thirteen  of  the  14  projects  were  structured  into  two 
semesters,  with  only  one  institution  (Penn  State)  conducting  a  single-semester  capstone  project 
course.  However,  one  (University  of  Maryland)  is  conducting  two  one-semester  capstone  courses 
(i.e.,  with  different  students).  Those  who  carried  over  two  semesters  did  so  in  one  of  two  ways: 
(1)  The  first  semester  was  an  introduction  to  fundamental  SE  concepts  and  processes  and  the 
second  was  devoted  to  the  development  of  the  capstone  design  project  or  (2)  both  semesters  were 
devoted  to  the  capstone  project  work. 

Student  Teams 


The  student  population  also  varied  in  terms  of  the  mix  of  graduate  students  and  undergraduates, 
as  illustrated  in  Eigure  5.  Nine  of  the  institutions  had  only  undergraduates  as  students,  two  had 
exclusively  graduate  students  and  three  had  a  mix.  In  addition,  two  institutions  had  graduate 
students  acting  as  teaching  assistants  or  providing  technical  help.  Student  teams  ranged  from  four 
to  seven  members.  Teams  met  during  class,  at  lab  sessions  at  some  institutions,  and  also 
communicated  through  a  number  of  non-face-to-face  channels,  including  e-mail,  telephone, 
videoconference,  weblogs,  and  on  16  collaborative  document  sharing  platforms.  Teams  generally 
submitted  weekly  progress  reports  and  prepared  final  project  presentations.  At  two  institutions, 
student  teams  included  distance  students  who  communicated  with  their  teams  via  their  university 
Blackboard  portal  or  the  Stevens  project- wide  Sakai  site.  In  half  the  universities,  students  chose 
their  own  teams;  in  the  rest,  faculty  did  so. 
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Formation  of  Student  Teams 


■  Student  self  selection 


■  Faculty  selection-  with 
students  matched  by  GPA 
or  other  facto's 

■  Ihpsti.dent  grotjpwac 
smalland  tnere  wa^  only 
one  team 

■  Faculty  selection  -  with 
sorestudont  input 
(based  on  proolcm  areal 


Figure  5 


Figure  6  shows  that  over  75  pereent  of  all  eapstone  student  teams  were  mixed  in  diseipline, 
allowing  faeulty  and  students  to  experience  working  in  a  context  resembling  that  of  professional 
systems  engineers: 


Student  Characteristics 


The  total  number  of  students  returning  surveys  was  294.  Of  those,  127  were  undergraduates,  124 
were  graduate  students,  and  1 1  were  postgraduates.  While  the  total  number  of  undergraduates 
and  graduate  students  was  nearly  equal  across  the  13  institutions,  a  closer  look  at  differences 
between  individual  institutions  shows  that  nearly  half  of  the  13  were  comprised  entirely  of 
undergraduates,  as  illustrated  in  Figure  7.  Four  institutions  had  graduate  students  (including 
postgraduate  students)  and  the  remaining  three  had  mixed  undergraduate  and  graduate 
populations.  However,  the  ratio  varied. 

Class  sizes  varied  widely  across  institutions,  ranging  from  a  low  of  3  or  4  students  to  48.  The 
average  class  size  was  20  (median  =  17,  SD  =  14). 
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Figure  7 


Figure  8  illustrates  that  most  of  the  undergraduates  were  in  their  senior  year  and  most  of  the  graduate 
students  were  in  their  first  year. 


All  Institutions  -  By  Class  year 
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Figure  8 


The  most  prevalent  engineering  discipline  among  students  across  all  institutions,  as  illustrated  in 
Figure  9,  was  SE  followed  by  Mechanical  Engineering,  Electrical  Engineering,  Computer 
Science/Software  Engineering,  and  Industrial  Engineering. 
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Figure  9 


Overall,  the  student  population  (n=264)  was  over  three-quarters  male  (77  percent),  with  a  small 
female  population  (16  percent)  and  a  small  percentage  of  students  selecting  not  to  report  their 
gender  (7  percent).  Figure  10  shows  the  gender  of  the  student  population. 


All  Institutions  -  By  Gender 
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Figure  10 


As  illustrated  in  Figure  11,  students’  reported  ethnicities  (n=264)  were:  White  (64  percent); 
Asian  (11  percent);  Black  or  African-American  (7  percent);  Hispanic/Latino  (5  percent); 
Hawaiian  or  Other  Pacific  Islander  (>1  percent);  and  unreported  (11  percent). 
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Figure  11 


Interest  in  SE  as  a  Career 


In  the  pre-survey,  less  than  half  (41  pereent)  of  all  students  reported  a  high  level  of  interest  in 
beeoming  a  systems  engineer,  while  18  pereent  reported  moderate  interest  and  25  pereent 
reported  little  interest;  15  pereent  were  not  sure  (Figure  12). 


Working  for  the  government  as  a  systems  engineer 

Approximately  the  same  pereentage  (45  pereent)  of  the  students  reported  they  had  a  high  interest 
in  working  for  the  government  as  a  systems  engineer,  while  20  pereent  said  they  had  a  moderate 
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interest  and  19  pereent  said  they  had  little  interest  in  doing  this;  13  percent  were  not  sure  (Figure 
13). 


All  Institutions  -  Interest  in  working  for 
the  government  as  systems  engineer 
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DoD  Mentors 


Among  the  Pis  who  had  collaborations  with  DoD  mentors,  very  different  levels  of  interaction 
with  them  were  reported  as  indicated  in  Figure  14.  Despite  efforts  on  the  part  of  the  DoD,  several 
had  not  been  assigned  or  connected  with  projects  at  the  time  of  submission  of  the  interim  reports. 
Thus  while  over  43  percent  of  the  Pis  reported  that  their  DoD  mentors  were  “very  involved,” 
another  50  percent  reported  not  having  a  mentor  or  not  yet  working  with  their  mentor: 
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Some  Pis  reported  that  their  mentors  were  playing  an  “aetive  role  in  meeting. .  .by  phone,  email 
and  VTC,”  while  others  reported  that  the  mentor  had  not  followed  up  with  the  students  after 
some  initial  eommunication.  Of  the  Pis  who  had  mentors,  one -third  reported  that  the  DoD 
mentors  communieated  at  least  weekly  (one  even  reported  daily  interaction),  22  percent  reported 
that  they  communicated  bi-weekly,  and  45  percent  reported  they  had  communicated  at  least 
several  times  a  semester  (Figure  15). 


Frequency  of  DoD  mentor-student 
interaction 

■  Weekly  "Biweekly  ■  A  few  times  during  the  semester 


Figure  15 

Like  RT-19  faculty,  DoD  mentors  facilitated  student  learning  in  a  variety  of  ways;  however,  their 
roles  differed  instructionally  depending  on  whether  they  also  served  as  clients,  as  they  did  for  50 
percent  of  participating  institutions.  Some  of  their  roles  included: 

•  Providing  feedback  on  student  projects  and  deliverables 

•  Providing  technical  expertise 

•  Attending  student  presentations 

•  Facilitating  field  trips  offsite  to  manufacturing  or  design  sites  where  students  could 
observe  engineering  processes  related  to  their  projects 

Mentors  with  the  highest  level  of  student  engagement  interacted  with  students  in  every  single 
activity  area  and  with  frequency,  while  others  provided  only  intermittent  correspondence.  Note 
that  mentors  could  engage  in  more  than  one  kind  of  interaction.  Figure  16  represents  the  level  of 
Mentor-Student  Interaction. 
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As  illustrated  in  Figure  17,  the  Pis  therefore  differed  in  their  evaluation  of  the  mentors’ 
eontributions  to  student  learning  and  engagement.  Note  that  mentors  eould  make  eontributions  in 
more  than  one  area: 
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Figure  17 


Industry  Mentors 


About  60  pereent  of  the  Pis  reported  not  having  an  industry  mentor.  Three  of  the  eight  who  did 
not  have  industry  mentors  reported  that  they  planned  to  work  with  one  during  the  spring 
semester  (Figure  18). 
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Participation  of  Industry  Mentors 

■  Industry  mentor  ■  No  industry  mentor 


Figure  18 

Industry  mentors  took  on  roles  similar  to  the  roles  played  by  DoD  mentors — as  clients, 
consultants,  or  SMEs.  Note  that  mentors  could  play  more  than  one  role: 


Industry  Mentors'  Roles 
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Pis  at  two  institutions  that  had  both  industry  and  DoD  mentors  reported  that  having  both  types  of 
mentors  benefited  the  students’  projects.  For  example,  one  PI  reported  that  the  DoD  mentor  acted 
as  a  client  while  also  offering  occasional  help  with  analysis  and  understanding  system 
requirements,  while  the  industry  mentor  acted  as  a  consultant  “educating  [students]  on  their 
technical  approach.”  The  other  PI  whose  team  included  both  types  of  mentors  described  the 
industry  mentor  as  “part  of  the  project  teams. .  .intimately  involved  with  [students’]  day  to  day 
progress  and  project  management  aspect,”  compared  with  the  DoD  mentor,  who  took  the 
customer’s  perspective  and  dealt  more  with  design  and  engineering  concerns. 
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At  another  institution,  however,  a  PI  expressed  dissatisfaetion  with  the  mentor’s  overlapping 
roles  of  client  and  project  advisor/consultant  and  argued  for  better  defined  responsibilities. 
Initially  regarded  by  students  as  a  “customer”  or  “major  stakeholder,”  the  DoD  mentor 
functioned  in  reality  as  a  subject  matter  expert  who  assisted  students  with  the  design  process. 

Discussion 

It  is  premature  at  this  stage  of  course  implementation  to  report  on  student  outcomes  with  the 
structure  or  content  of  the  courses  or  with  any  particular  strategies  or  course  materials.  However, 
the  Pis’  interim  reports  provide  some  insights  into  preliminary  lessons  learned  about  course 
objectives  and  implementation.  These  observations  fall  into  several  categories: 

1.  Challenges  teaching  the  broad  topic  of  SE  to  non-SE  majors  under  time  constraints. 

2.  Challenges  with  equivalent  grading  policies  in  multi-disciplinary  teams,  particularly  where  SE 
was  an  overlay  to  an  existing  multi-disciplinary  team  structure. 

3.  Challenges  with  content-domain-specific  problem  areas  and  with  finding  meaningful  ways  for 
other  disciplinary  majors  to  contribute. 

4.  Motivating  external  mentors  to  bring  authentic  professional  experiences  to  the  learning 
experience. 

5.  SE  content  modules  provide  opportunities  to  bring  non-SE  majors  to  a  common 
understanding.  These  have  been  implemented  with  varying  frequency,  durations,  and  numbers 
across  several  projects. 

6.  Efforts  to  provide  specific  disciplinary  expertise  (internal  and  external)  to  infuse  sufficient 
content  knowledge  into  student  teams  such  that  students  are  able  to  focus  on  the  bigger  SE 
competencies. 

It  is  also  not  clear  at  this  stage  the  extent  to  which  external  funding  has  created  entirely  new 
materials  or  simply  (and  in  some  cases,  substantially)  enhanced  existing  courses.  This  is  an  area 
for  further  investigation. 

Next  Steps  and  Future  Research 

As  indicated  throughout  this  paper,  the  preliminary  observations  and  findings  represent  only  a 
snapshot  of  the  richness  of  the  14  SE  Capstone  courses  that  have  been  and  are  being 
implemented  by  pilot  institutions  in  most  cases,  mid-way  through  the  implementation  of  their 
courses.  At  this  point,  no  comprehensive  analysis  of  student  learning  has  been  conducted  at  this 
stage  of  course  implementation.  Eurther  analysis  and  later  papers  will  aim  to  connect  the  course 
content  and  organization,  including  materials  created  by  faculty  as  well  as  the  contributions  of 
external  mentors,  with  impact  on  student  learning  of  SE  content,  their  interest  in  SE  careers,  and 
their  interest  in  DoD  problem  areas  and  careers. 
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As  noted,  this  research  has  been  undertaken  in  order  to  inform  the  development  of  a  larger 
capacity-building  and  scale-up  effort  that  could  substantially  increase  the  SE  workforce  available 
to  DoD  and  industry  in  the  next  decade  and  beyond.  This  research  will  capture  methods, 
strategies,  and  tools  that  have  led  to  desired  student  outcomes.  Further  research  to  translate  these 
findings  into  methods,  tools,  and  processes  that  can  be  operationalized  in  new  universities  has 
been  proposed.  Anticipating  a  scale-up  effort  involving  additional  universities  (n=20-100),  the 
lessons  learned  from  the  SE  Capstone  Project  may  be  applied  across  a  diverse  set  of  institutions, 
with  varying  capacity  in  SE. 
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